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TITLE OF THE INVENTION 

WAVEFORM GENERATING METHOD, PERFORMANCE DATA PROCESSING 
METHOD , WAVEFORM SELECTION APPARATUS, WAVEFORM DATA 
RECORDING APPARATUS, AND WAVEFORM DATA RECORDING AND 
REPRODUCING APPARATUS 

BACKGROUND OF THE INVENTION 

Field of the Invention 

This invention relates to a waveform generating 
method, a performance data processing method, a waveform 
selection apparatus, a waveform data recording apparatus, 
and a waveform data recording and reproducing apparatus 
for use in making performance using recorded musical 
instrument tones and singing tones. 

Description of the Related Art 

Conventionally, there is known a waveform memory 
tone generator. The waveform memory tone generator 
generates musical tones by reading out waveform data 
stored in a waveform memory according to event data such 
as note-on and note-off. In reading waveform data from 
the waveform memory, different waveform data are selected 
from the waveform memory according to pitch information 
and touch information contained in note-on events . 

On the other hand, an electronic musical instrument 
called a sampler is also conventionally known. The 
sampler samples and records waveforms of monotones being 
performed (for example, a pitch C3 for five seconds) and 
generates musical tones using the recorded waveform data. 
In this case, the user sets original keys representing 
tone pitches of respective ones of the sampled waveform 
data, for the respective sampled waveform data. A range 
where the waveform data is to be used, an amount of pitch 
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shift during sounding at a predetermined pitch, and the 
like are determined according to the original keys . To 
determine the original keys, pitches are sampled from the 
sampled waveform data and the original keys are 
5 automatically set for the waveform data correspondingly 
to the sampled pitches (refer to Japanese Laid-Open 
Patent Publication (Kokai) No. 07-325579, for example) . 

A phrase sampler has been also proposed which 
samples phrase waveforms being performed, divides each of 
10 the sampled phrase waveforms into a plurality of partial 
waveform data according to an envelop level sampled from 
the sampled phrase waveform data to make performance 
while changing timing and pitch of each partial waveform 
data. 

15 If tones being actually generated by performance are 

sampled and recorded and musical tones are generated 
using the recorded waveform data as is the case with the 
conventional sampler, tones are generated with poorer 
expression than tones actually generated by performance 

20 by musical instruments. In this case, it is impossible 
to obtain real musical tones even if waveform data is 
selected according to pitch information and touch 
information . 

Further, according to the conventional sampler, 
25 since each waveform data for use in performance is 

recorded in the form of monotones, tones generated based 
upon the recorded waveform data are unnatural as is 
different from tones produced during performance of music. 
Specifically, the player needs to sequentially perform 
3 0 and record necessary monotones in a sampler during 

recording, but the player cannot easily perform only a 
monotone without getting nervous as compared with 
performance of a phrase. Particularly when a single tone 
is sung as a vocal, a voice may be uttered poorly or in a 
3 5 falsetto tone. Since it is difficult to record monotones 
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in natural tone color, natural tones cannot be easily 

generated by performance by the sampler. 

Further, when setting original keys for the recorded 

monotones, a person must determine the original keys or 
5 pitches must be sampled to determine the original keys. 

He or she, however, must be experienced to determine the 

original keys, and therefore, every person cannot 

determine the original keys. On the other hand, sampling 
M" the pitches requires complicated arithmetic operations, 

10 and moreover, the sampled pitches may cause one tone 
31 being generated by performance to be incorrectly 

2 recognized as multiple tones due to a change in pitch 

Lfl over time or may cause any of harmonic tones existing in 

some tones rich with harmonic components being generated 
p 15 by performance to be incorrectly determined as 

5j fundamental tones. Thus, the sampled pitches cannot 

t always be correct. That is, even if the pitches are 

0 sampled, a person must finally confirm the sampled 

pitches . 

2 0 Further, the conventional phrase sampler does not 

sample a pitch from waveform data when dividing a phrase 
waveform into partial waveforms. More specifically, a 
position at which a phrase waveform is divided into 
partial waveforms does not necessarily correspond to a 

25 point in change of pitch, and it is therefore impossible 
to set original keys for respective ones of partial 
waveform data. Further, to set original keys for partial 
waveform data with the dividing position being regarded 
as a point of change in pitch, the user needs to set the 

30 original keys or sample pitches from waveform data as 

mentioned above. This requires complicated operations as 
described above. 
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SUMMARY OF THF, INVENTION 



It is therefore a first object of the present 
invention to provide a waveform generating method, a 
performance data processing method, a waveform selection 
apparatus, a waveform data recording apparatus, and a 
recorded waveform reproducing apparatus, which are 
capable of generating expressive musical tones. 

It is a second object of the present invention to 
provide a waveform generating method, a performance data 
processing method, a waveform selection apparatus, a 
waveform data recording apparatus, and a recorded 
waveform reproducing apparatus, which make it possible to 
record natural tones, divide recorded waveform data at 
points of change in pitch, and automatically assigning 
property information on pitch and the like to the divided 
waveform data. 

It is a third object of the present invention to 
provide a waveform generating method, a performance data 
processing method, a waveform selection apparatus, a 
waveform data recording apparatus, and a recorded 
waveform reproducing apparatus, which are capable of 
obtaining natural performance tones. 

To attain the first object, the present invention 
provides a waveform generating method comprising the 
steps of storing a plurality of partial waveforms in a 
partial waveform memory, storing property information on 
respective ones of the partial waveforms stored in the 
partial waveform memory, in a property information memory, 
retrieving the property information memory according to 
inputted sounding control information to read out a 
partial waveform having property information 
corresponding to the sounding control information, 
processing the readout partial waveform according to the 
property information and the sounding control information, 
and generating a waveform corresponding to the sounding 
control information. 
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To attain the first object, the present invention 
also provides a performance data processing method 
comprising the steps of storing property information on 
respective ones of a plurality of partial waveforms in a 
5 property information memory, comparing characteristics of 
respective ones of notes included in performance data 
with the property information stored in the property 
information memory to detect an optimum partial waveform 
for a characteristic of each of the notes, assigning 
^ 10 designation data for designating the detected partial 

^ waveform to each of the notes, and storing performance 

% i data having the designation data assigned thereto. 

X To attain the first object, the present invention 

£0 further provides a waveform generating method for use in 

^ 15 reproducing the performance data having the designation 

data assigned thereto stored by the above-mentioned 
performance data processing method, comprising the steps 
of storing a plurality of partial waveforms in a partial 
ft! waveform memory, reading out a partial waveform from the 

2 0 partial waveform memory according to the designation data 
assigned to each of the notes to be reproduced when 
reproducing the notes having the designation data 
assigned thereto according to the performance data, and 
generating a waveform corresponding to each of the notes 

2 5 based upon the read out partial waveform. 

To attain the first object, the present invention 
yet further provides a waveform generating method 
comprising the steps of storing performance information 
corresponding to real-time performance, and generating 

3 0 accompanying tones by automatic performance or automatic 

accompaniment according to a tempo clock, reproducing 
performance information according to the tempo clock, and 
generating waveforms corresponding to performance events 
performed in real time to accompaniment of the generated 
3 5 accompanying tones, according to the performance events 
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and the reproduced performance information. 

To attain the first object, the present invention 
further provides a waveform selection apparatus 
comprising a partial waveform memory that stores a 
5 plurality of partial waveforms, a database that stores 
property data representing characteristics of two tones 
corresponding to each of the partial waveforms stored in 
the partial waveform memory and consisting of a tone 
corresponding to each of the partial waveforms and a 

10 preceding tone, and a retrieving device that retrieves 
the database according to characteristic data of two 
tones consisting of an inputted present tone to be 
generated by performance and an inputted preceding tone 
to be generated by performance before the inputted 

15 present tone to extract at least one partial waveform 

having property data close to the characteristic data of 
the inputted two tones as a partial waveform for sounding 
the inputted present tone to be generated by performance. 
To attain the first object, the present invention 

2 0 further provides a waveform selection apparatus 

comprising a partial waveform memory that stores a 
plurality of partial waveforms, a database that stores 
property data representing characteristics of two tones 
corresponding to each of the partial waveforms stored in 
25 the partial waveform memory and consisting of a tone 
corresponding to each of the partial waveforms and a 
following tone, and a retrieving device that retrieves 
the database according to characteristic data of two 
tones consisting of an inputted present tone to be 

3 0 generated by performance and a following tone to be 

inputted and generated by performance before the inputted 
present tone to extract at least one partial waveform 
having property data close to the characteristic data of 
the inputted present tone and the following tone to be 
35 inputted as a partial waveform for sounding the inputted 
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present tone to be generated by performance. 

According to the waveform generating method of the 

present invention, it is possible to generate waveforms 

using the optimum partial waveforms correspondingly to 
5 sounding control information, and hence generate 

expressive musical tones. 

Further, according to the performance data 

processing method of the present invention, it is 
I* possible to enable the optimum partial waveform data for 

Ji* 10 characteristics of each note in the performance data to 

fH be selected in advance and assigned to each note in the 

"2 performance data. The use of such performance data 

in eliminates the necessity of selecting the optimum partial 

waveform data during waveform generation. 
Pi : 15 Further, according to the waveform generating method 

Hi of the present invention, even when musical tones are 

% generated by real-time performance, it is possible to 

Q control characteristics of musical tones generated by 

ft! performance being currently made according to following 

20 performance to be subsequently made, by utilizing 

performance information corresponding to real-time 

performance . 

Further, by selecting one or more partial waveforms 
having property data close to characteristic data of the 

25 following two tones, a present tone to be currently 
generated by performance and a preceding tone to be 
generated by performance before the present tone or a 
following tone to be generated by performance next to the 
present tone, the optimum partial waveforms can be 

30 selected as partial waveforms for generating musical 
tones by performance . 

To attain the second object, the present invention 
provides a waveform data recording apparatus comprising 
an automatic performing device that reproduces 

35 performance data relating to phrases to be recorded, a 



waveform recording device that records phrase waveforms 
representing tones generated by performance based on 
tones generated by reproduction of the performance data 
by the automatic performing device, and a waveform data 
5 processing device that extracts data of the phrase 
waveforms recorded by the waveform recording device 
according to characteristic information on notes of the 
performance data to divide the data of the phrase 
M: waveforms into partial waveform data corresponding to 

10 respective ones of the notes. 
s f Preferably, the waveform recording device records 

y- the phrase waveforms in synchronism with performance 

timing of the automatic performing device. 
03 Also preferably, the waveform data processing device 

JU 15 assigns the characteristic information on the notes 

fyi corresponding to the partial waveform data as property 

information to the partial waveform data. 

Preferably, the waveform data recording apparatus 
HI comprises a device that creates a tone color set from a 

20 partial waveform pool in which selected partial waveform 
data selected from the partial waveform data is pooled, 
and a partial waveform management database composed of 
the property information assigned to respective ones of 
the selected partial waveform data pooled in the partial 
25 waveform pool. 

To attain the third object, the present invention 
also provides a recorded waveform data reproducing 
apparatus comprising a storage device that stores a tone 
color set from a partial waveform pool in which selected 
3 0 partial waveform data selected from the partial waveform 
data is pooled, and a partial waveform management 
database composed of property information assigned to 
respective ones of the selected partial waveform data 
pooled in the partial waveform pool, a detecting device 
3 5 that retrieves the partial waveform management database 
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in the tone color set according to characteristic 
information on respective notes in performance data to 
detect optimum partial waveform data for each of the 
notes from the partial waveform pool, a designation data 
5 inserting device that embeds designation data for 

designating the optimum partial waveform data detected by 
the detecting device to each of the notes into the 
performance data, and a reproducing device that 
automatically reproduces the performance data in which 

10 the designation data is embedded by the designation data 
inserting device, according to the optimum partial 
waveform data designated by the designation data. 

To attain the third object, the present invention 
further provides a recorded waveform data reproducing 

15 apparatus comprising a storage device that stores a tone 
color set from a partial waveform pool in which selected 
partial waveform data selected from the partial waveform 
data is pooled, and a partial waveform management 
database composed of property information assigned to 

2 0 respective ones of the selected partial waveform data 

pooled in the partial waveform pool, a detecting device 
that retrieves the partial waveform management database 
in the tone color set according to information on 
respective performance event data that have occurred, to 
25 detect optimum partial waveform data for each of the 

performance event data from the partial waveform pool, 
and a reproducing device that reproduces the performance 
event data according to the optimum partial waveform data 
detected by the detecting device. 

3 0 To attain the third object, the present invention 

yet further provides a recorded waveform data reproducing 
apparatus comprising a storage device that stores a tone 
color set from a partial waveform pool in which selected 
partial waveform data selected from the partial waveform 
35 data is pooled, and a partial waveform management 
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database composed of property information assigned to 
respective ones of the selected partial waveform data 
pooled in the partial waveform pool, a detecting device 
that retrieves the partial waveform management database 
in the tone color set according to characteristic 
information on respective notes in performance data read 
in advance to detect optimum partial waveform data for 
each of the notes from the partial waveform pool, and a 
reproducing device that is responsive to occurrence of 
performance event data corresponding to respective ones 
of the notes in the performance data, for reproducing 
performance tones corresponding to the respective ones of 
the notes according to the optimum partial waveform data 
detected by the detecting device. 

According to the present invention constructed as 
above, it is possible to enable performance to be made 
while listening to tones generated by performance by the 
automatic performing device, and thus enable the player 
to make performance relaxedly and make recording of 
natural tones . By extracting the thus recorded phrase 
waveform data according to the automatically reproduced 
performance data, the phrase waveform data can be divided 
into partial waveform data corresponding to notes of the 
performance data. In this case, the dividing position is 
made more accurate by recording tones generated by 
performance in synchronism with the performance timing of 
the automatic performing device. Further, the 
characteristic information such as the pitch, length, 
intensity of a note corresponding to the partial waveform 
data obtained by the division can be assigned as property 
information to the corresponding partial waveform data. 
Further, desired partial waveform data can be selected 
from the partial waveform data obtained by the division 
and combined with the property information thereof to 
provide the tone color set for performance. 



11 



To make automatic performance based on the tone 
color set, the partial waveform management database of 
the tone color set is retrieved according to the pitch, 
length, intensity, etc. of each note in performance data 
5 for automatic performance to detect the optimum partial 
waveform data, and designation data for designating the 
detected partial waveform data to the note is embedded in 
the performance data. This enables automatic performance 
to be made with natural tones based on the processed 
10 performance data. 

Further, to make real-time performance based on the 
tone color set, the partial waveform management data of 
the tone color set is retrieved according to information 
ffl on generated performance event data to detect the optimum 

^ 15 partial waveform, and the detected partial waveform data 

-1 is used to reproduce performance tones of the performance 

O event data, to thereby enable performance to be made with 

natural tones . 

fy Further, to make performance based on the tone color 

2 0 set, the partial waveform management database of the tone 

color set is retrived according to the pitch, length, 
intensity, etc. of each note in performance data read in 
advance to detect the optimum partial waveform data, and 
upon occurrence of performance event data corresponding 
25 to respective ones of the notes in the performance data 
when the same part as the performance data is performed, 
performance tones corresponding to the respective notes 
are reproduced according to the detected optimum partial 
waveform data. This enables detection of conditions of 

3 0 following tones to be reproduced and hence enables 

performance to be made with more natural tones. 

The above and other objects, features, and 
advantages of the invention will become more apparent 
from the following detailed description taken in 
35 conjunction with the accompanying drawings. 
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BRIEF DESCRIPTION OF THE DRAWINGS 



FIG. 1 is a diagram showing the hardware 
5 construction of a waveform data recording and reproducing 
apparatus that executes a waveform generating method and 
a performance data processing method according to an 
embodiment of the present invention; 

FIGS . 2A and 2B are diagrams showing the 
10 relationship between automatically-reproduced performance 
data and phrase waveform data, which are synchronized, in 
the waveform data recording and reproducing apparatus, 
wherein : 

yj FIG. 2A shows the performance data ; and 

L. 15 FIG. 2B shows the phrase waveform data; 

i_ FIGS. 3A and 3B are diagrams showing modes of 

recording waveform data by the waveform data recording 
7h and reproducing apparatus according to the embodiment, 

SU wherein : 

20 FIG. 3A shows the case where a player is listening 

to automatically-perf ormance tones via a headphone; and 
FIG. 3B shows the case where a player is listening 
to automatically-perf ormance tones via a speaker; 

FIG. 4 is a schematic diagram showing a phrase 
25 waveform data processing process and a property 

information assigning process carried out by the waveform 
data recording and reproducing apparatus according to the 
embodiment ; 

FIG. 5 is a diagram showing the structure of data 
3 0 recorded in the waveform data recording and reproducing 
apparatus according to the embodiment; 

FIG. 6 is a diagram showing the structure of a 
performance tone color set in the waveform data recording 
and reproducing apparatus according to the embodiment; 
3 5 FIG. 7 is a diagram showing the detailed structure 
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of a partial waveform record in the performance tone set 
in the waveform data recording and reproducing apparatus 
according to the embodiment; 

FIG. 8 is a diagram showing the structure of a 
partial waveform ID of a partial waveform record in the 
performance tone color set in the waveform data recording 
and reproducing apparatus according to the embodiment; 

FIG. 9 is a flow chart showing the procedure for 
producing the tone color set in the waveform data 
recording and reproducing apparatus according to the 
embodiment ; 

FIG. 10 is a flow chart showing a performance data 
processing process carried out by the waveform data 
recording and reproducing apparatus according to the 
embodiment ; 

FIG. 11 is a flow chart showing a process in a step 
S12 in the flow chart of FIG. 10; 

FIG. 12 is a flow chart showing a pattern comparing 
process in the flow chart of FIG. 11; 

FIG. 13 is a flow chart showing a note-on event 
process carried out by the waveform data recording and 
reproducing apparatus according to the embodiment; 

FIG. 14 is a process in a step S2 0 in the note-on 
event process of FIG. 13; 

FIG. 15 is a flow chart showing an automatic 
performance starting process carried out by the waveform 
data recording and reproducing apparatus according to the 
embodiment ; 

FIG. 16 is a flow chart showing a tempo interrupting 
process carried out by the waveform data recording and 
reproducing apparatus according to the embodiment during 
automatic performance; and 

FIG. 17 is a flow chart showing a note-on process 
carried out by the waveform data recording and 
reproducing apparatus according to the embodiment during 
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real-time performance made in combination with automatic 
performance . 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 

5 

The present invention will now be described in 
detail with reference to the drawings showing embodiments 
thereof . 

FIG. 1 shows the hardware construction of a waveform 
10 data recording and reproducing apparatus that executes a 
waveform generating method and a performance data 
processing method according to an embodiment of the 
present invention. 

In the waveform data recording and reproducing 
15 apparatus 1 in FIG. 1, a CPU 10 is a central processing 
unit that executes various kinds of programs to control 
recording and reproducing operations carried out by the 
waveform data recording and reproducing apparatus 1 . A 
timer 11 is a timer that indicates elapsed time during 

2 0 operation and generates a timer interrupt at 

predetermined intervals, and is used for time management 
during automatic performance. A ROM 12 is a read only 
memory that stores programs for executing a dividing 
process and a property information assigning process 
25 carried out by the CPU 10 during recording, programs for 
executing a performance data processing process, a note- 
on event process, and the like carried out by the CPU 10 
during reproduction, and various kinds of other data. A 
RAM 13 is a random access memory that serves as a main 

3 0 memory in the waveform data recording and reproducing 

apparatus 1, and as a work area and the like for the CPU 
10 is set in the RAM 13. 

A display 14 shows various kinds of information 
during recording and reproduction, and an operating 
3 5 element 15 is operated to perform various operations 
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during recording and reproduction and may include a 
keyboard. A tone generator 16 generates musical tones 
based on performance data comprised of multiple phrases 
which are automatically performed during recording, and 
5 the tone generator 16 may be any one of an FM tone 

generator, a PCM tone generator, a harmonic synthesis 
tone generator, and so forth. A mixer 17 mixes musical 
tones generated by the tone generator 16 and musical 
M» tones reproduced by a reproducing circuit 24 described 

10 later, and transmits the mixed tones to a DAC 18. The 
§1 DAC (Digital to Analog Converter) 18 converts musical 

tone data supplied from the mixer 17 into an analog 
fjl musical tone signal, and transmits the analog musical 

W tone signal to a sound system 19. The sound system 19 

m 15 amplifies and sounds the musical tone signal supplied 

fU from the DAC 18. 

'*% A microphone 20 receives musical instrument tones 

+■ 

O generated by performance by a player or singing tones, 

iy i.e. melodies sung by a player, and a phrase waveform 

2 0 signal representing the received musical instrument tones 
or singing tones is transmitted to an ADC (Analog to 
Digital Converter) 21. The ADC 21 converts the waveform 
signal representing the supplied tones into digital 
phrase waveform data, and the resulting phrase waveform 

2 5 data is recorded in a HDD (Hard Disk Drive) 23 under the 

control of a recording circuit 22. As described later, 
the recorded phrase waveform data is divided into partial 
waveform data according to performance data reproduced by 
the tone generator 16 during recording, and the resulting 

3 0 partial waveform data has property information assigned 

thereto. Such dividing process and property information 
assigning process are carried out by the CPU 10 by 
executing programs for implementing these processes. 
Further, partial waveform data to be used is selected 
35 from the partial waveform data, and a performance tone 
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color set is produced from property information assigned 
to the selected partial waveform data. The reproducing 
circuit 24 retrieves a partial waveform management 
database for a designated tone color in the tone color 
5 set, and acquires the optimum partial waveform data from 
the property information to generate performance waveform 
data. The generated performance waveform data is 
output ted as sounds from the sound system 19 via the 
mixer 17 and the DAC 18. 

10 An MIDI interface 25 receives an MIDI signal that is 

output ted from an MIDI keyboard or the like conforming to 
the MIDI, so that the reproducing circuit 24 can generate 
performance waveform data as described above according to 
event data in the MIDI signal and the sound system 19 can 

15 sound performance tones. Another interface 26 is a 

communication interface for communication networks such 
as a LAN (Local Area Network) , a public telephone network, 
and the Internet, and is connected to a server computer 
via a communication network so that desired performance 

2 0 data can be downloaded from the server computer. 

Reference numeral 27 denotes a bus line for use in 
sending and receiving signals between devices. 

Referring next to FIGS. 2 - 6, a description will be 
given of a process for recording phrase waveform data and 
25 dividing it into partial waveform data in the waveform 
data recording and reproducing apparatus 1 according to 
the present embodiment. 

To record phrase waveform data by the waveform data 
recording and reproducing apparatus 1 according to the 

3 0 present embodiment, an automatic performing device 31 is 

supplied with performance data 3 0 composed of a plurality 
of phrases to make automatic performance as shown in FIG. 
3A. The phrases include notes having characteristic 
information such as pitch desired for recording. The 
3 5 automatic performing device 31 is comprised mainly of the 
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CPU 10 and the tone generator 16 in FIG. 1. The CPU 10 
executes an automatic performance program to generate 
tone generator parameters according to performance data 
readout from the ROM 12 or the RAM 13 . The tone 
5 generator parameters are supplied to the tone generator 
16 to cause the performance waveform data generated by 
the tone generator 16 to be sounded from a headphone 3 6 
connected to the sound system 19 via the mixer 17 and the 
y= DAC 18. A player 32 mounts thereon the headphone 3 6 so 

10 that he or she can listen to tones being generated by 
performance via the headphone 36. It should be noted 
SI that sequence data, which is, for example, in the SMF 

(Standard MIDI File) format or the like and comprised of 
£0 phrases for one or more parts including performance data 

* m 15 for one or more parts desired to be recorded or 

fy performance data for other parts, is prepared in advance 

y and stored as performance data in the RAM 13 or the like. 

^ While listening to tones being generated by 

fU performance based on the performance data, the player 3 2 

2 0 performs a musical instrument, not shown, or sings a song 

according to the performance tones. Since the player 3 2 
performs a musical instrument or sings a song while 
listening to the performance tones, the performance tones 
or singing tones can be natural . Phrase waveforms 
25 representing the performance tones or singing tones are 
supplied to a waveform recording device 33 via a 
microphone 20, and are sampled into digital phrase 
waveform data 34 and recorded on a phrase-by-phrase basis. 
On this occasion, a clock synchronization line 35 

3 0 synchronizes clocks of the automatic performing device 31 

and the waveform recording device 33, so that phrase 
waveform data can be recorded in synchronism with 
performance timing. This synchronization is carried out 
by using the same clock for a sampling clock of the 
3 5 waveform recording device 3 3 and an operating clock of 
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the automatic performing device 31, and making the 
automatic performance starting timing and the recording 
starting timing coincide with each other or storing a 
difference between the two kinds of timing. This ensures 
5 the recorded phrase waveform data 34 and the 

automatically reproduced performance data to be 
synchronized over all sections thereof. Thus, even a 
change in the tempo of the performance data would not 
, affect the synchronization at all. 

Q 10 FIGS. 2A and 2B show the outline of the relationship 

~™ between the automatically reproduced performance data and 

Sj the phrase waveform data. The performance data is 

=P represented by notes on a score as shown in FIG. 2A, and 

=v FIG. 2B shows phrase waveforms recorded by performance or 

s 15 singing in accordance with automatic performance tones 

51 based on the performance data listened to via the 

O headphone 36. In this case, a quarter note ™E" in the 

jj' performance data corresponds to a waveform a, a quarter 

note "A" in the performance data corresponds to a 
20 waveform b, an eighth note "F" in the performance data 

corresponds to a waveform c, an eighth quarter "E" in the 
performance data corresponds to a waveform d, a quarter 
note "C" in the performance data corresponds to a 
waveform e, and a note "F" in the performance data 
25 corresponds to a waveform f. Due to the synchronization 
as described above, each waveform corresponds to the 
performance timing of the performance data, and has a 
tone length corresponding to a note. Further, the tone 
color of each waveform is natural since the player makes 
30 performance while listening to the performance tones 

based on the performance data, though this is not seen 
from the waveforms . 

The waveform recording device 33 is comprised of the 
CPU 10, the ADC 21, the recording circuit 22, and the HDD 
35 23 in FIG. 1. Under the control of the CPU 10, the 



19 



waveform recording device 33 converts phrase waveforms 
representing performance tones or singing tones inputted 
from the microphone 2 0 into the digital phrase waveform 
data 34. The phrase waveform data 34 is written into a 
5 predetermined storage area of the HDD 23 by the recording 
circuit 22 . It should be noted that automatic 
performance may be carried out in any of modes described 
below. The first automatic performance mode is a solo- 
s' part performance based on performance data comprised only 
S 10 of a part desired for recording (recording part) in a 
5 phrase. The second automatic performance mode is an all- 
%f part performance based on performance data comprised of a 
jt plurality of parts including a part desired for recording 
T in a phrase. The third automatic performance mode is a 
L, 15 minus-one performance based on performance data comprised 
jTj of one or more parts except for a part desired for 

recording in a phrase, or comprised of the above- 
£. mentioned other parts . 

HI The waveform data recording and reproducing 

20 apparatus 1 according to the present embodiment may 

record the phrase waveforms using the arrangement in FIG. 
3B. In the arrangement shown in FIG. 3B, the player 32 
is listening to tones being generayed by automatic 
performance via a speaker 37 instead of the headphone 36. 

25 A description will now be given only of parts of the 
arrangement different from FIG. 3A. Automatic 
performance tones outputted from the automatic performing 
device 31 are sounded via the speaker 3 7 and audited by 
the player 32. On this occasion, the automatic 

30 performance tones sounded via the speaker 37 are picked 
up by the microphone 20, and thus, a speaker tone- 
removing device 38 removes the automatic performance 
tones picked up by the microphone 20. It should be noted 
that the speaker tone removing device 3 8 is supplied with 

35 automatic performance tones from the automatic performing 
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device 31, and the automatic performance tones are 
removed by inverting the phase of the automatic 
performance tones and carrying out time 
adjustment/amplitude adjustment. 
5 Since the waveform recording device 33 samples tones 

generated by performance or singing by the player 32 with 
the automatic performance device 31 set in any one of the 
above described modes of automatic performance, the 
E = . recorded phrase waveform data always includes performance 

O 10 data (corresponding performance data) for waveform 

division corresponding to parts desired for recording 
S| (recording parts) . 

i*! Accordingly, as shown in FIG. 4, a dividing device 

:~r 41 divides phrase waveform data 40 into partial waveform 

s _ 15 data 43 corresponding to respective notes in performance 

S'J data 44 according to the automatically reproduced 

£f performance data 44. A property information-assigning 

£ device 42 then assigns to the resulting partial waveform 

■fg data 43 property information representing properties such 

2 0 as pitch, tone length and intensity of the respective 
notes while referring to characteristic information on 
the respective notes in the corresponding performance 
data 44. The CPU 10 executes a dividing process program 
and a property information assigning process program to 

2 5 cause the dividing device 41 to divide the phrase 

waveform data 40 into the partial waveform data 43 and 
cause the property information-assigning device 42 to 
assign property information to the divided partial 
waveform data 43 . 

3 0 In this case, since the phrase waveform data 40 is 

synchronous with the performance data 44, it is possible 
to divide the phrase waveform data 40 into the partial 
waveform data 43 only according to sounding timing or the 
like of the performance data 44. Further, the dividing 
35 position of the phrase waveform data 40 may be corrected 
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by frequency analysis (formant analysis) . More 
specifically, a start position of a formant in the 
partial waveform data 43 is searched for based upon a 
provisional dividing position provisionally determined 
5 according to the performance data 44, and the detected 
start position is determined as the dividing position. 
This enables the phrase waveform data 40 to be divided at 
more musically accurate positions as compared with a 

p. dividing method based only upon results of analysis of 

jgj 10 the phrase waveform data 40. 

J" To be more specific, the dividing method based only 

%J upon the performance data 44 assumes that the 

~- automatically reproduced performance data 44 and the 

09 phrase waveform data 40 are completely synchronized. 

15 Thus, according to data of each note contained in the 

U 

fU performance data for a recording part, a portion of the 

? =g phrase waveform data 40 for a time range corresponding to 

==£= 

the note (from start txmxng of a tone corresponding to 
the note (note-on) to the time point of attenuation of 
20 the tone or to the start of a tone of the next note after 
the start of releasing of the tone (note-off) ) can be 
extracted or cut out as each piece of partial waveform 
data 43. 

Since the phrase waveform data 40 is obtained by 
25 sampling the performance made by a person, however, the 

start timing of some notes in the performance data 44 for 
the recording part does not necessarily coincide with the 
start timing of waveforms corresponding to the notes and 
may be different from the latter. 
3 0 To accurately extract or cut out the partial 

waveform data 43, with reference to start timing of each 
note in the performance data for the recording part, 
waveforms of the phrase waveform data 40 in sections just 
before and after the above start timing (e.g. sections 
35 over several seconds before and after the start timing) 
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are analyzed, and the start timing of the note in the 
phrase waveform data 40 is detected based upon the 
analysis results, to thereby correct the dividing 
position . 

5 A waveform analyzing method for use in correcting 

the dividing position can be based upon formant analysis 
and FFT analysis. In the formant analysis, with 
reference to start timing of each note in the performance 
1^ data for the recording part, an LPC (Linear Prediction 

Q 10 Coding) coefficient is calculated from a correlation 

Zl function of waveform data in sections just before and 

■%| after the above start timing of the phrase waveform data 

T: 40. The LPC coefficient is then converted into a formant 

m 

gj parameter to find a formant in the phrase waveform data 

" 15 40. The start timing of the note is detected from a 

rising position of the formant corresponding to the note. 

CS This enables the phrase waveform data 40 to be divided 

_r-;; 

into the partial waveform data 43 for respective notes at 
f!j musically accurate positions. 

2 0 In the FFT (Fast Fourier Transform) analysis, with 

reference to start timing of each note in the performance 
data for the recording part, sections just before and 
after the above start timing of the phrase waveform data 
40 are subjected to the fast fourier transformation while 
25 shifting a time window. The loci of a fundamental tone 

and multiple harmonic tones corresponding to the note are 
detected, and the start timing is detected based on a 
rising position in the detected loci of the fundamental 
tone and harmonic tones . This method enables the partial 

3 0 waveform data 43 to be divided into the partial waveform 

data 43 for respective ones of notes at musically 
accurate positions. Although in the above description, 
as the waveform analyzing method, the formant analysis 
and the FFT analysis were given, other analyzing methods 
35 such as pitch analysis and envelope analysis may be 
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adopted to correct the dividing position. 

Upon completion of the dividing process, a property 
information assigning process is carried out to assign 
property information to respective partial waveform data 
5 obtained by the division. In the property information 
assigning process, the partial waveform data 43 
corresponding to respective notes in the performance data 
44 have assigned thereto property information 
Lg, corresponding to character information of the notes. The 

~ 10 property information includes one or more kinds of 

j^: information among pitch information used as original keys, 

SI intensity information used as original intensity, tone 

;fj length information used as original tone length, 

preceding tone information and following tone information 
!L 15 used for selecting partial waveform data. It should be 

noted that the preceding tone information is information 
indicating whether the pitch of the preceding tone is 
s£ higher or lower than that of the present tone or 

f|j information on a difference in pitch between the 

2 0 preceding tone and the present tone, information 

indicating whether the intensity of the preceding tone is 
higher or lower than that of the present tone or 
information on a difference in intensity between the 
preceding tone and the present tone, information 
25 indicating whether the tone length of the preceding tone 
is longer or shorter than that of the present tone or 
information on a difference in tone length between the 
preceding tone and the present tone, and other 
information relating to the relationship between 

3 0 preceding tone and the present tone. In this case, the 

preceding tone may be in plurality, for example, two 
tones before the present tone. The following tone 
information is information relating to the relationship 
between the following tone and the present tone in the 
3 5 case where the preceding tone in the preceding tone 
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information is replaced, by the following tone. Further, 
if the performance data includes musical symbols 
representing slur, trill, and crescendo, such musical 
symbols may be included in the property information. The 
5 property information is used as a reference in selecting 
partial waveform data, and is used as a parameter in 
processing partial waveform data. It should be noted 
that partial waveform data corresponding to the desired 

y s property information can be obtained by recording with 

10 notes corresponding to the desired property information 

g4 being included in performance data. 

Si As shown in FIG. 4, the dividing device 41 divides 

J the phrase waveform data 40 into the partial waveform 

CP data 43, and the property information assigning means 42 

JU, 15 assigns the property information to the divided partial 

ffj waveform data 43 . The partial waveform data 43 obtained 

y by the division and the assigned property information are 

= stored as one recorded data together with the performance 

fp data in the HDD 23. Since the phrase waveform data 40 is 

2 0 recorded for each type of musical instruments and each 

vocal tone color, the data is recorded for each tone 
color. FIG. 5 shows the structure of the data recorded 
in the HDD 23. As shown in FIG. 5, the recorded data is 
comprised of performance data composed of phrases which 
25 were automatically performed, phrase waveform management 
information for managing a location in the storage area 
where phrase waveform data corresponding to performance 
data for each phrase is stored, partial waveform 
management information composed of a start position and 

3 0 end position in the storage area of partial waveform data 

divided from phrase waveform data and property 
information of the partial waveform data, and a phrase 
waveform 1, a phrase waveform 2, ... corresponding to 
performance data for respective phrases . Data of each 
3 5 phrase waveform is divided into a plurality of partial 
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waveform data in the above -described method, and details 
of partial waveform data divided from the phrase waveform 
1 is illustrated in FIG. 5 as an example. Specifically, 
the phrase waveform 1 is divided into six partial 
5 waveform data: a partial waveform 1-1, a partial waveform 
1-2, a partial waveform 1-6. 

The recorded data in FIG. 5 is obtained by dividing 
phrase waveform data recorded for each performance data 
y. and assigning property information to the resulting 

10 partial waveform data. The phrase waveform data is 

recorded for each type of musical instruments or each 
\1 vocal tone color . Thus , the recorded data is produced 

jfj for each tone color. Therefore, to obtain performance 

Oil waveform data for a certain tone color, corresponding 

8 m 15 partial waveform data is read out from the recorded data 

rlj according to performance event data and used. The 

^ partial waveform data in the recorded data in FIG. 5, 

PI however, may include a plurality of partial waveform data 

ffj with the same property information such as pitch, and 

2 0 unnecessary partial waveform data. Therefore, partial 

waveform data to be used is selected from the recorded 
data to produce a performance tone color set for use in 
performance, based on the selected partial waveform data. 
To produce the tone color set, selecting information 
25 indicating which partial waveform data is to be selected, 
tone color parameters, and the like are contained in the 
recorded data as tone color management information, and 
the tone color set is produced according to the tone 
color management information. For this reason, the 

3 0 recorded data in FIG. 5 can be called a tone color set 

producing tool. The tone color management information 
includes selecting information for selecting partial 
waveform data according to the user's request, tone color 
parameters set by the user, and so forth. 
3 5 FIG. 6 shows an example of the structure of data in 
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the tone color set. The performance tone color set shown 
in FIG. 6 is comprised of tone color data for each tone 
color such as "violin 1", "male voice 4", "trumpet 
2", ... . Every tone color data has the same structure. 
5 For example, the structure of tone color data of "male 
voice 4". As shown in FIG. 6, the tone color data is 
comprised of a header, a partial waveform management 
database, tone color parameters, and a partial waveform 
^ pool. In the partial waveform pool, partial waveform 

•^ 10 data selected from a plurality of phrase waveform data 

: - according to the tone color management information in the 

M recorded data in FIG. 5 is pooled as a partial waveform 1, 

a partial waveform 2, a partial waveform 3, ... as shown 
m in FIG. 6. In this case, if partial waveform data 

^ 15 corresponding to many pitches are selected according to 

the tone color management information, it is possible to 
D finely set a range to which are assigned the partial 

X: waveform data. Also, the tone color management 

M 

111 information may be such information as enables selection 

2 0 of partial waveform data including the same pitch 

information but different tone length information or 
intensity information, or partial waveform data including 
different information on the difference in pitch between 
the preceding tone and the present tone or different 

2 5 information indicating whether the pitch of the preceding 

tone is higher or lower than that of the present tone. 

The partial waveform management database in the tone 
color set produced in the above-mentioned manner has 
recorded therein information indicating a start position 

3 0 and an end position of partial waveform data selected 

according to the tone color management information in the 
storage area for the partial waveform pool and property 
information of the partial waveform data. The tone color 
parameters included in the tone color management 
35 information are directly stored in a tone color parameter 



27 



storage area, and an envelop parameter and a filter 
coefficient for a corresponding tone color are recorded 
as the tone color parameters . 

A plurality of tone color data, which are each 
5 produced for each tone color in the tone color set, can 
be produced from one recorded data by selecting different 
combinations of partial waveform data from the recorded 
data in FIG. 5. The waveform data recording and 
reproducing apparatus 1 according to the present 
zz 10 invention is capable of making automatic performance and 

S real-time performance using the tone color set thus 

Nf produced . 

=p FIG. 7 shows the structure of data in the partial 

© waveform management database which stores property 

15 information and the like of partial waveforms selected 
ffj according to the tone color management information stored 

=5f in the partial waveform pool. The partial waveform 

fj management database is comprised of record information 

PJ such as a partial waveform 1 record, a partial waveform 2 

20 record, a partial waveform 3 record, a partial waveform 4 
record, . . . representing characteristics of respective 
partial waveforms such as the partial waveform 1, the 
partial waveform 2, the partial waveform 3, the partial 
waveform 4, ... stored in the partial waveform pool. 
25 Every record information has the same data structure, and 
FIG. 7 shows the details of data structure of the partial 
waveform 3 record. Specifically, the record information 
represented by the partial waveform 3 record includes a 
partial waveform ID (PW (Partial Wave) ID) , a partial 
3 0 waveform name (PWNAM) , a partial waveform producing date 
(PWDAT) , and a partial waveform author (PWAUT) . As shown 
in FIG. 8, the partial waveform ID is comprised of a 
phrase waveform ID (more significant 10 bits) and a 
partial waveform ID (less significant 6 bits) . It should 
3 5 be noted that the phrase waveform ID is an ID of a phrase 
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waveform from which the partial waveform has been cut out, 
and the partial waveform number is a number indicating 
when the partial waveform was cut out from the phrase 
waveform. The partial waveform ID thus constructed can 
5 specify a phrase waveform from which a partial waveform 
is cut out, and makes it possible to confirm the type of 
the phrase waveform from which a partial waveform has 
been cut out, by listening to tones represented by the 
phrase waveform when the user selects the partial 
r*j 10 waveform. 

O Further, each record information includes a variety 

01 

r;: of information on tone pitch (PWPIT) , intensity (PWINT) , 

45 tone length (PWLEN) of partial waveform data, a pitch 

£ ratio (PWPFR) representing the relationship in pitch 

g* 15 between the present tone and the following tone, an 

^ intensity ratio (PWFIR) between the present tone and the 

"■=? following tone, a tone length ratio (PWFLR) between the 

M present tone and the following tone, a pitch ratio 

O 

(PWBPR) representing the relationship in pitch between 

2 0 the preceding tone and the present tone, an intensity 

ratio (PWBIR) between the preceding tone and the present 
tone, and a tone length ratio (PWBLR) between the 
preceding tone and the present tone. Among these 
information, the information relating to the ratios may 

25 alternatively be replaced by information on differences. 

Further, each record information includes a start 
address (PWSA) , end address (PWEA) , beat address (PWBA) , 
and loop address (PWLA) of a corresponding partial 
waveform in the storage area for the partial waveform 

30 pool. In this case, the start address and the end 

address are essential information, but the beat address 
and the loop address should not always be included in the 
record information. It should be noted that the beat 
address represents a beat position (a position where a 

3 5 beat is felt) in the partial waveform, and the beat 
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position is set to correspond to the timing of a note in 
the performance data when the partial waveform data is 
used. To use the beat address is favorable in the case 
where the start position of each performed note is not 
5 clear when performance data is softly performed. The 

loop address is an address of a loop waveform in the case 
where a constant portion with a small change in a partial 
waveform is replaced by a repeated loop waveform. It 
should be noted that even if the partial waveform is 
10 shortened by looping, this does not cause a change in 

Q 

m tone length information of the partial waveform. Thus, 

\l the tone length (PWLEN) information is information that 

? r does not indicate the length of waveform data but 

CEi indicates the length of a note when the tone of the 

L, 15 waveform data is generated by performance. 

fir The partial waveform management database includes 

the extent of slur (PWSLR) , the depth of vibrato (PWVIB) , 
and other information. 

It should be noted that the partial waveform 
20 management database may be separated into a database that 
stores address information such as start addresses, end 
addresses, etc. of partial waveforms, and a database that 
stores the rest of information on partial waveforms for 
use in retrieving partial waveforms. In this case, the 

2 5 latter database for retrieval is only required for 
selecting partial waveforms, and the processing carried 
out by CPU 10 can be reduced by integrating the former 
address information database in the tone generator 16. 

Referring next to a flow chart of FIG. 9, a 

3 0 description will now be given of the procedure for 
producing the tone color set for use in performance shown 
in FIGS. 6-8 using the tone color set producing tool 
having the data structure as shown in FIG. 5. It should 
be noted that the user who produces the tone color set 

3 5 carries out this procedure. 
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First, the user who produces the tone color set 
prepares and stores performance data comprised of a 
plurality of phrases including at least a high-pitch note 
required for a recording part in the performance data 
storage area in FIG. 5 (step SI) . On this occasion, the 
performance data may be comprised of phrases including 
notes having characteristic information required for 
recording, i.e. information on tone pitch, tone length, 
velocity, preceding tone information, difference in pitch 
between the preceding tone and the present tone, etc. 
Then, while musical tones generated by automatic 
performance by the waveform data recording and 
reproducing apparatus 1 based upon the performance data 
comprised of a plurality of phrases thus prepared are 
supplied to the player, phrase waveforms generated by 
performance or singing by the player are recorded in 
necessary ones of the phrase waveform storage areas of 
the waveform data recording and reproducing apparatus 1 
shown in FIG. 5 (step S2 ) . 

The waveform data recording and reproducing 
apparatus 1 divides the recorded phrase waveform data 
into partial waveform data according to the performance 
data in the recorded data, and assigns property 
information relating to notes corresponding to the 
partial waveform data obtained by the division to the 
partial waveform data. The start position and end 
position of each partial waveform data obtained by the 
division and the property information assigned to the 
partial waveform are stored as partial waveform 
management information for each partial waveform in the 
partial waveform management information storage area in 
FIG. 5 (step S3) . According to operating element 
operation by the user, the partial waveform data are 
selectively combined, and tone color parameters such as 
an envelope parameter are prepared. Selecting 
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information indicating the selected partial waveform data 
and the prepared tone color parameters are stored as tone 
color management information in the tone color management 
information storage area in FIG. 5 (step S4) . 
5 According to the tone color data for one tone color 

completed in the steps SI to S4, tone color data for one 
tone color in the tone color set in FIG. 6 is prepared 
and stored in a storage area in a manner described below. 
y, First, partial waveform data selected from a plurality of 

10 partial waveform data stored in the phrase waveform 
=y storage areas in FIG. 5 according to the selecting 

%J information in the tone color management information is 

copied in a partial waveform pool for the tone color in 
CO FIG. 6, and the start position and end position of each 

^ 15 copied partial waveform data and property information 

fy including the ID, name, author, etc. of each copied 

0 partial waveform data are stored in the partial waveform 

q management database in FIG. 6. The tone color parameters 

fU in the tone color management information in FIG. 5 are 

20 copied in a tone color parameter storage area for the 
tone color in FIG. 6. Finally, header information 
indicating the tone color type, tone color name, tone 
color data capacity, etc. of the tone color are stored in 
the header area in FIG. 6 to complete the tone color data 
25 in the tone color set for use in performance. 

It should be noted that in the step S3 , after a 
position of dividing the phrase waveform data into the 
partial waveform data and property information to be 
assigned are automatically determined, the user may 
3 0 arbitrarily correct the dividing position and the 
property information. 

To apply the performance tone color set in FIG. 6 to 
automatic performance in the waveform data recording and 
reproducing apparatus 1, performance data is processed in 
3 5 advance prior to the automatic performance. The 
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performance data is arbitrary performance data which is 
desired for automatic performance by the user. FIG. 10 
shows a flow chart showing a performance data processing 
process when the tone color set is applied to automatic 
performance, and a description thereof will be given 
hereinbelow. 

Upon start of the performance data processing 
process in FIG. 10, performance data to be processed is 
designated in a step S10. The user designates 
performance data desired for automatic performance among 
performance data stored in the ROM 12 or the RAM 13 . In 
the following description, the performance data is 
monophonic performance data composed of one part in order 
to simplify the description. In the next step Sll, a 
pointer is set to a top note in the designated 
performance data. In the next step S12, search 
information in the partial waveform management database 
for the designated tone color in the tone color set is 
searched according to characteristic information such as 
the pitch, intensity (velocity) , length, pitch ratio 
between the present tone and the following tone, 
intensity ratio between the present tone and the 
following tone, and tone length ratio between the present 
tone and the following tone for a note indicated by the 
pointer (at this time, the top note in the performance 
data) , to detect the optimum partial waveform data. 

In the next step S13, designation information for 
designating the optimum partial waveform data detected 
correspondingly to the note indicated by the pointer is 
embedded in the performance data. Specifically, 
designation information for designating partial waveform 
data such as a meta event and a system exclusive message 
is inserted just before performance event data 
corresponding to the note. In the next step S14, the 
pointer is moved to the next note, and in a step S15, it 
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is determined whether a note is present or not at a 
position to which the pointer has been moved. If a note 
is present, the program returns to the step S12 . Thus, 
the process from the steps S12 to S15 is then repeatedly 
5 carried out, so that designation information for 
designating the optimum partial waveform data 
corresponding to one note is embedded in the performance 
data each time the process is carried out. After 
y, completion of embedment of designation information for 

10 designating the optimum partial waveform data 

corresponding to the last note in the performance data, 
%J it is determined in the step S5 that no note is present, 

£ and then the process proceeds to a step S16. In the step 

m S16, the performance data in which the designation 

15 information on the partial waveform data is embedded in 
fij the above-mentioned manner is stored with a different 

name. This completes the performance data processing 
I- process. 

HI If the thus processed performance data is designated 

2 0 and automatically performed by the waveform data 

recording and reproducing apparatus 1, in start timing of 
each note in the performance data, partial waveform data 
specified by the designation information embedded just in 
front of a note -on event of the note is read out from the 
25 HDD 23 and transmitted to the reproducing circuit 24 

together with tone color parameters corresponding to the 
note-on event. In this manner, the performance data is 
automatically performed to generate musical tones 
corresponding to respective notes in the specified 

3 0 partial waveform data. 

Although the designation information for designating 
the partial waveform data is embedded as a meta-event and 
a system exclusive message, the designation information 
may be embedded as other kinds of events . 
35 Further, if performance data desired for automatic 
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performance is comprised of a plurality of parts, the 
process in FIG. 10 may be carried out for only one 
desired part among the plurality of parts, or may be 
carried out for desired two or more parts . 

Further, the process in FIG. 10 is carried out not 
only for monophonic performance data but also for 
performance data having a plurality of notes overlapping 
at the same time. However, if partial waveform data is 
desired to be selected according to characteristic 
information indicating whether the pitch of the preceding 
tone or the following tone is higher or lower than that 
of the present tone, a difference in pitch between the 
preceding tone or the following tone and the present tone, 
or the like, all of performance data to be processed must 
be monophonic . 

FIG. 11 is a flow chart showing the details of the 
partial waveform selecting process carried out in the 
step S12 in the performance data processing process in 
FIG. 10. 

In the partial waveform selecting process in FIG. 11, 
present tone data such as present tone pitch (PPIT) , 
present tone intensity (PINT) , and present tone length 
(PLEN) are acquired in a step S10. Next, following tone 
data such as following tone pitch (FPIT) , following tone 
intensity (FINT) , and following tone length (FLEN) are 
acquired in a step S31. Further, preceding tone data 
such as preceding tone pitch (BPIT) , preceding tone 
intensity (BINT) , and preceding tone length (BLEN) are 
acquired in a step S32. A following tone pitch ratio 
(FPR) and a preceding tone pitch ratio (BPR) are then 
calculated according to the following equations (1) and 
(2) : 

FPR= (FPIT-PPIT) /PPIT —(1) 
BPR= (BPIT-PPIT) /PPIT "-(2) 

Further, a following tone intensity ratio (FIR) and 
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a preceding tone intensity ratio (BIR) are calculated 
according to the following equations (3) and (4): 

FIR= (FINT-PINT) /PINT —(3) 

BIR= (BINT-PINT) /PINT -"{4) 

Further, a following tone length ratio (FLR) and a 
preceding tone length ratio (BLR) are calculated 
according to the following equations (5) and (6) : 

FLR= (FLEN-PLN) /PLEN "-(5) 

BLR= ( BLEN- PLEN ) / PLEN ■ • ■ ( 6 ) 

Further, other parameters are detected in a step S36. 
In this detection, the extent of slur is detected and the 
vibration width of pitch bend data is detected as the 
depth of vibrato according to a period of time in which 
the present tone overlaps the following tone, variations 
of intensity in a plurality of preceding and following 
tones, and the like. 

The information such as the present tone pitch 
(PPIT) , the present tone intensity (PINT) , the present 
tone length (PLEN) obtained in the step S3 0, the 
information such as the following tone pitch ratio (FPR) , 
the preceding tone pitch ratio (BPR) , the following tone 
intensity ratio (FIR) , the preceding tone intensity ratio 
(BIR) , the following tone length ratio (FLR) , and the 
preceding tone length ratio (BLR) calculated in the steps 
S3 3 to S3 5, and the information obtained in the step S3 6 
constitute characteristic information on a tone of a note 
indicated by the pointer. Accordingly, the pattern of 
this characteristic information is compared with that of 
characteristic information on each partial waveform in a 
step S37 to select a partial waveform having the closest 
characteristic information. The ID of the selected 
partial waveform is stored in an SPWID register. This 
completes the partial waveform selecting process in the 
step S12. 

FIG. 12 is a flow chart showing the pattern 
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comparing process carried out in the step S37 . 

In the pattern comparing process, the present tone 
pitch (PPIT) , the present tone intensity (PINT) , and the 
present tone length (PLEN) corresponding to the note 
indicated by the pointer are compared with a partial 
waveform tone pitch (PWPIT) , a partial waveform tone 
intensity (PWINT) , and a partial waveform tone length 
(PWLEN) of each partial waveform, to thereby select a 
limited number of candicate partial waveforms according 
to the result of the comparison, to reduce the number of 
calculations . An example of the method for selecting the 
candidate partial waveforms will now be given. First, 
partial waveforms with a difference between the partial 
waveform pitch (PWPIT) and the present tone pitch (PPIT) 
lying in a range AP, a difference between the partial 
waveform tone intensity (PWINT) and the present tone 
intensity (PINT) lying in a range Al, and a difference 
between the partial waveform tone length (PWLEN) and the 
present tone length (PLEN) lying in a range AL are 
selected. In this case, if the number of selected 
partial waveforms is too small, the ranges AP, Al , and AL 
are widened (the conditions are relaxed) to select 
partial waveforms again according to the widened ranges . 
If, however, the number of selected partial waveforms is 
sufficient, the limiting process is finished. 
Alternatively, the candidate partial waveforms may be 
selected as follows. First, distances (PND) representing 
differences between the present tone and respective ones 
of all partial waveforms are calculated in order to find 
the similarity between the present tone and the 
respective ones of partial waveforms in pitch, intensity, 
and tone length. Next, n partial waveforms are selected 
beginning from a partial waveform with the smallest 
distance representing the similarity. 

After the candidate partial waveforms are selected 
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in this manner, the distances relating to the pitch, 
intensity, and tone length of the respective ones of the 
selected partial waveforms and the tone of the note 
indicated by the pointer are calculated in a step S41 
5 according to the following equation (7) . If, however, 

the distances (PND) are calculated using the equation (7) 
in the step S40, the step S41 is skipped. 

PND2=ap (PPIT-PWPIT) 2+bp ( PINT-PWINT) 2+cp ( PLEN-PWLEN) 2 

{* *'" (7) 

C3 10 In the equation (7) , symbols ap, bp, and cp 

represent coefficients for PND calculation. 
%| In a step S42 , distances (FND) relating to the 

+ f following tone pitch, following tone intensity, and 

U1 

ff following tone length ratio in respective ones of the 

f 15 selected candidate partial waveforms 1 and the pitch 

jT ratio, intensity ratio, and tone length ratio of the 

IP following tone to the present tone of the note indicated 

% by the pointer are calculated according to the following 

:\ equation (8) : 

2 0 FND2=af (FPR-PWFPR) 2+bf (FIR-PWFIR) 2+cf (FLR-PWFLR) 2 

•••(8) 

In the equation (8) , symbols af , bf , and cf 
represent coefficients for FND calculation. 

Further, in a step S43 , distances (BND) relating to 
25 the preceding tone pitch, preceding tone intensity, and 
preceding tone length ratio in respective ones of the 
selected candidate partial waveforms and the pitch ratio, 
intensity ratio, and tone length ratio of the preceding 
tone to the present tone of the note indicated by the 

3 0 pointer are calculated according to the following 

equation (9) : 

BND2=af (BPR-PWBPR) 2+bf (BIR-PWBIR) 2+cf (BLR-PWBLR) 2 
■•■(9) 

In the equation (9), symbols ab, bb, and cb 
3 5 represent coefficients for BND calculation. 
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In the next step S44, a total distance (TOTALD) 
representing the total similarity between respective ones 
of the selected candidate partial waveforms and the tone 
of the note indicated by the pointer is calculated 
5 according to the following equation (10) using the 

distances (PND) , the distances (FND) , and the distances 
(BND) calculated in the steps S41 to S43 : 

TOTALD=at • PND + bt • FND + ct ■ BND •••(10) 
5 , In the equation (10), symbols at, bt, and ct 

Q 10 represnet coefficients for TOTALD calculation, and these 

; « coefficients have a relationship of at>ct>bt. 

iy Upon calculation of the total distance (TOTALD) 

«P relating to the respective ones of the selected candidate 

m 

ffi. partial waveforms in the step S44, a partial waveform 

s 15 with the smallest total distance (TOTALD) calculated in 
Jt; the step S45 is selected and an ID thereof is stored in 

p the SPWID register. This completes the pattern comparing 

f process. 

P 

ry It should be noted that in the step S12 of the 

2 0 performance data processing process, any one partial 

waveform is selected in whatever state of present data of 
a note indicated by the pointer in the performance data . 
However, the minimum conditions may be set for the degree 
of agreement or similarity between present data and 
25 property data, and then, if no property data of any of 

the partial waveforms satisfy the minimum conditions, it 
may be determined that "there is no corresponding partial 
waveform" . If it is determined that there is no 
corresponding partial waveform, the user is preferably 

3 0 warned or an ordinary waveform memory tone generator 

preferably sounds tones instead of partial waveforms . 
Further, instead of selecting only one partial waveform, 
a second candidate partial waveform, a third candidate 
partial waveform, . . . may be automatically selected in 
35 advance according to an instruction from the user, and 
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from among these candidate partial waveforms one partial 
waveform may be selected. 

Referring next to a flow chart of FIG. 13, a 
description will be given of a note-on event process in 
the case where the tone color set is applied to real-time 
performance in the waveform data recording and 
reproducing apparatus 1 according to the present 
embodiment . 

If a note-on event is generated by depressing the 
keyboard included in the operating element 15 or a note- 
on event is supplied via the MIDI interface 25, the 
present note-on event process is started to select the 
optimum partial waveform data by searching property 
information in the partial waveform management database 
for a designated tone color in the tone color set 
according to information such as pitch and intensity in 
the note-on event generated in the step S2 0. In this 
case, the selection of the partial waveform data may be 
based on combinations of information such as (1) the 
pitch of the note-on, (2) the pitch and intensity of the 
note-on, (3) the pitch of the note-on and information 
indicating whether the pitch of the preceding tone is 
higher or lower, (4) the pitch of the note-on and a 
difference in pitch between the preceding tone and the 
note-on, and (5) the intensity of the note-on and the 
preceding tone. 

Upon selection of the optimum partial waveform 
corresponding to the note-on event, the reproducing 
circuit 24" assigns a sounding channel in a step S21, and 
in a step S22, information of the selected partial 
waveform data, tone color parameters, etc. are set to the 
assigned sounding channel. On this occasion, the pitch 
shift amount is also determined according to a difference 
in pitch between the note-on event and the selected 
partial waveform data. In the next step S23, the note-on 
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is transmitted to the assigned sounding channel, and the 
reproducing circuit 24 reproduces a musical tone waveform 
according to the partial waveform data and tone color 
parameters and the pitch shift amount which have been 
5 thus set. The reproduction of the musical tone waveform 
corresponding to the note-on event completes the note-on 
event process, and if a note-on event based on real-time 
performance is generated again, the present note-on event 
s , process is started again to repeatedly carry out the 

Q 10 process of reproducing a musical tone waveform 

corresponding to the note-on event, 
y It should be noted that in real-time performance, 

4* the length of a tone to be generated by performance is 

not known until the a corresponding note-off occurs, and 
a 15 it is therefore impossible to carry out time-base control 

^ of the partial waveform data according to the tone length. 

=1: Accordingly, as is the case with conventional ordinary 

2% waveform memory tone generators, a constant section of 

m the musical tone waveform immediately following an attack 

2 0 thereof is provided with a loop section for performing 

loop reproduction so that the length of the musical tone 
is controlled by providing an attenuation envelop 
according to loop reproduction + note-off. 

Further, in a variation in the case where the tone 
25 color set is applied to real-time performance, if a delay 
of several seconds to several dozens of seconds in 
sounding is allowed from the generation of the note-on to 
the sounding, a performance event occurring during the 
delay time period is stored in a buffer so that 

3 0 information such as the tone length of the note-on and 

the preceding tone can also be used in selection of 
partial waveform data. 

FIG. 14 is a flow chart showing the partial waveform 
selecting process carried out in the step S20 of the 
3 5 note-on event process in FIG. 13. 
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In the partial waveform selecting process in the 
step S20, the present tone pitch (PPIT) and the present 
tone intensity (PINT) of a noted-on present tone are 
acquired in a step S51. However, the present tone length 
5 (PLEN) cannot be acquired since the tone is being noted- 
on. Next, preceding tone data such as the preceding tone 
pitch (BPIT) , preceding tone intensity (BINT) , and 
preceding tone length (BLEN) of the preceding tone in the 
previous note-on are acquired in a step S52 . In the next 
q 10 step S53, a pitch ratio (BPR) of the acquired present 

O tone pitch (PPIT) to the preceding tone pitch (BPIT) is 

Is calculated according to the above-mentioned equation (2) . 

45 Further, in a step S54, an intensity ratio (BIR) of the 

I r present tone intensity (PINT) to the preceding tone 

s 15 intensity (BINT) is calculated according to the above- 

mentioned equation (4) . Further, in a step S55, the 
detection of other parameters is carried out, that is, 
4S the extent of slur is detected and the vibration width of 

the pitch bend data is detected as the depth of vibrato 
2 0 according to a period of time in which the present tone 
overlaps the preceding tone, the variation of the 
intensity, and the like. 

The pattern of the characteristic data of the noted- 
on present tone pitch (PPIT) and present tone intensity 

2 5 (PINT) , the calculated preceding tone pitch ratio (BPR) 

and preceding tone intensity ratio (BIR) obtained in the 
steps S51 to S55 are compared with that of the 
characteristic data of each partial waveform to select a 
partial waveform with the closest characteristic data. 

3 0 The ID of the selected partial waveform is stored in the 

SPWID register. This completes the partial waveform 
selecting process in the step S20. It should be noted 
that the pattern comparing process can be carried out in 
the same manner as the pattern comparing process in FIG. 
35 12. However, since there is no following tone, the 
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patterns are compared based on data relating to the 
present tone and the preceding tone. More specifically, 
the distance (FND) relating to the following tone is not 
calculated, and the distance (PND) relating to the 
5 present tone is calculated using the coefficient ap set 
to zero. According to the calculated distance (PND) and 
distance (BND) , the total distance (TOTAL.D) is calculated 
to select a partial waveform with the smallest total 
y, distance (TOTALD) , and the ID of the selected partial 

3; 10 waveform is stored in the SPWID register. 

- Incidentally, the tone color set may be applied to a 

%J combination of automatic performance and real-time 

irt performance in the waveform data recording and 

ffl; reproducing apparatus 1 according to the present 

8 15 embodiment. In this case, musical tone waveforms based 

fi on performance data processed by the performance data 

C3 processing process in FIG. 10 are generated in the 

5.1 automatic performance, and musical tone waveforms are 

fll generated by carrying out the note-on event process in 

2 0 FIG. 13 in the real-time performance in which the player 
makes performance according to the automatic performance. 
In this combination, if the real time performance is made 
in accordance with the automatic performance such as 
automatic rhythms, automatic accompaniment, and 

2 5 accompaniment parts, a tempo clock can be obtained from 

the automatic performance. Accordingly, if performance 
data for a real-time performance part performed by the 
player is prepared, he or she makes performance according 
to the real-time performance part, and this enables 

3 0 detection of conditions of following events such as note- 

on events and note-off events in the real-time 
performance. Therefore, in the partial waveform 
selecting process in the step S20 of the note-on event 
process in FIG. 13, a partial waveform can be selected 
35 using information on the tone length of a present note-on 
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and information on a tone to be generated by performance 
next, as well. Incidentally, in the case where automatic 
performance is made according to performance data 
composed of sixteen parts, if the first part is a real- 
5 time performance part, the automatic performance 

generates musical tones for the second to sixteenth parts. 
On the other hand, the first part is used for the 
selection of a partial waveform described above. 

As mentioned above, in the case of automatic 
r~i 10 performance, a tempo clock can be obtained from the 

O automatic performance. More specifically, only in 

01 

■ % s automatic performance, a tempo txmer generates a tempo 

45 interrupt at time intervals corresponding to the tempo. 

~ A tempo counter counts up by a predetermined value at 

s 15 every tempo interrupt. A present counter value POS of 

the tempo counter indicates an advancement position of 
\ automatic performance in performance data. This position 

<$> does not indicate an address position on a memory storing 

Si performance data, but indicates a position based on a 

2 0 tempo clock representing which clock of which beat in 

which measure. The performance data includes a real-time 
performance part performed by the player, and the present 
position of the real-time performance part is also 
determined in synchronism with automatic performance of 
25 other parts. 

FIG. 15 is a flow chart showing an automatic 
performance starting process in which the tempo counter 
is initialized. 

Upon start of the automatic performance starting 

3 0 process, performance data for automatic performance is 

specified in a step S60. The value POS of the tempo 
counter is then initialized in a step S61. A first event 
position in the specified performance data is then 
determined in a step S62, and the tempo timer is started 
35 in a step S63 . This starts automatic performance to 
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cause the tempo timer to be incremented, and when the 
determined event position is reached, a corresponding 
event is reproduced. 

FIG. 16 is a flow chart showing a tempo interrupting 
5 process for updating the tempo counter. 

If a tempo interrupt occurs, the tempo interrupting 
process is started to cause the value POS indicated by 
the tempo counter to increase by a predetermined value at 
y, every tempo interrupt in a step S7 0. It is then 

y 10 determined in a step S71 whether the present time point 

p- corresponds to an event position or not. If it is 

determined that the present time point corresponds to an 
event position, the process proceeds to a step S72 
ft} wherein an event at the event position is reproduced, and 

JL, 15 upon completion of the reproduction, a next event 

y 

HI position is determined in a step S72, and the process 

M; proceeds to a step S74. In this case, if the event is an 

event in real-time performance, the event is not 
fp reproduced. If it is determined in the step S71 that the 

2 0 present time point does not correspond to an event 
position, the process proceeds directly to the step S74 
wherein it is determined whether the present time point 
corresponds to an end position or not. If it is 
determined that the present time point corresponds to the 

25 end position, the process proceeds to a step S75 wherein 
the tempo timer is stopped to terminate the tempo 
interrupting process. On this occasion, the automatic 
performance is terminated. If it is determined in the 
step S74 that the present time point does not correspond 

3 0 to the end position, the tempo interrupting process is 
terminated. 

FIG. 17 is a flow chart showing a note-on process 
for the real-time performance made in combination with 
the automatic performance. 
3 5 When a note-on of the real-time performance is 



45 



detected, the present note-on process for the real-time 
performance is started to store a note number of the 
detected note-on in a PPIT register and store the 
velocity in a PINT register in a step S80. Then, 
5 preceding tone data of preceding tone pitch (BPIT) , 
preceding tone intensity (BINT) , and preceding tone 
length (BLEN) are acquired from the real-time performance 
in a step S81. Further, in a range in proximity to the 
value POS of the tempo counter indicating the present 
10 performance position, a note-on event for a real-time 
Q performance part in the performance data corresponding to 

t\ the detected note-number (PPIT) and velocity (PINT) is 

«p detected in a step S82 . Data of note-on tone length 

(PLEN) is acquired based on the tone of the detected 
a 15 event in a step S83 . In this case, a note-off event 

corresponding to the event (note-on event) is detected 
■p= from the performance data for the real-time performance 

45 part to obtain tone length information. 

Pi 

i'-i Then, following tone data of following tone pitch 

20 (FPIT) , following tone intensity (FINT) , and following 
tone length (FLEN) is acquired based on the following 
tone in the performance data of the detected event. In 
the next step S85, a partial waveform selecting process 
is carried out based on the data acquired from the 

25 performance data according to the acquired note-on data 
and the data acquired from the performance data based on 
the note-on data. In this partial waveform selecting 
process, the process from the steps S33 to S37 of the 
partial waveform selecting process in FIG. 11 is carried 

3 0 out. This selects a partial waveform with characteristic 
data closest to characteristic data of the note-on tone. 
Upon selection of the optimum partial waveform 
corresponding to the note-on tone, the reproducing 
circuit 24 assigns a sounding channel in a step S86, and 

35 information on the selected partial waveform, tone color 
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parameters, and the like are set to the assigned sounding 
channel in a step S87 . In this case, the pitch shift 
amount is also set based on the difference in pitch 
between the note number (PPIT) of the detected note-on 
5 and the selected partial waveform. 

The note-on is then transmitted to the assigned 
sounding channel in a step S88, and the reproducing 
circuit 24 reproduces a musical tone waveform based on 
y. the set partial waveform, tone color parameters, and 

?3 10 pitch shift amount. Upon reproduction of the musical 

tone waveform corresponding to the note-on, the note-on 

EH 

%J process for the real-time performance is terminated. If 

4* a note-on of real-time performance is generated again, 

the note-on process is started again to repeatedly carry 
= 15 out the above-described process of regenerating a musical 

tone waveform corresponding to the note-on event. 
O Although in the note-on process for real-time 

% performance in FIG. 17, the tone pitch (PPIT), intensity 

flj (PINT) , preceding tone pitch (BPIT) , preceding tone 

2 0 intensity (BINT) , and preceding tone length (BLEN) are 

acquired from the real-time performance, data obtained 
from performance data for a real-time performance part 
may be used for partial waveform selection. By using 
such data, a partial waveform to be selected upon each 
25 note-on in real-time performance can be selected in 

advance prior to the real-time performance. However, for 
the control of parameters for sounding, the real-time 
performance data is directly used. 

It should be noted that if all notes of pitch 

3 0 required for multiple sampling in which the keyboard 

range is divided into predetermined ranges and sampling 
waveforms are assigned to the predetermined ranges are 
included in phrases of performance data, all partial 
waveform data of the required pitch can be recorded as 
35 part of the recorded data. By doing so, performance can 
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be made based on partial waveform data obtained by the 
multiple sampling immediately after the performance of 
the phrases is completed. On this occasion, all the 
notes of required tone length and performance intensity 
5 may be included in the phrases of the performance data. 

Further, it may be arranged such that all partial 
waveform data obtained by dividing phrase waveform data 
can be used as tone color sets. Then, the procedure for 
creating selected tone color sets can be omitted, 

10 enabling performance to be immediately started. 

Further, various kinds of information may be used 
for other purposes than their original purposes . For 
example, in the case where a singing voice is sampled, 
"intensity information" may be used as "phoneme 

15 information" for discriminating the phoneme of the sound. 
Specifically, the intensity = 60 is assigned as 
information representing a phoneme "ah-ah-ah-", the 
intensity = 61 is assigned as information representing 
"ra-ra-ra-" , and the intensity = 62 is assigned as 

2 0 information representing a phoneme "du-du-du-" . If the 

"intensity information" is used as the "phoneme 
information" as mentioned above, by grouping partial 
waveform data according to the "intensity information ", 
groups of partial waveform data corresponding to 
25 respective identical phonemes can be collected. 

Further, performance data to be processed by the 
performance data processing process may be used directly 
as corresponding performance data serving as performance 
data for use in recording. This enables partial waveform 

3 0 data acquired by the division to be attached to the 

performance data after the recorded waveform data is 
divided into the partial waveform data, which simplifies 
the optimum partial waveform data detecting process. 
Further, by editing respective notes of the 
35 performance data to which is attached the partial 
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waveform data, the recorded phrase waveform data can be 
edited indirectly. 

Further, the performance data processing method 
according to the present invention enables the optimum 
partial waveform data for characteristics of each note in 
the performance data to be selected in advance and 
assigned to each note in the performance data. The use 
of such performance data eliminates the necessity of 
selecting the optimum partial waveform data during 
waveform generation. 



